At a hearing of the New York City Council, recently departed employees of Anthropic, OpenAI and Google DeepMind described a reckless culture at the firms, saying the biggest tech companies are racing to create technology they do not understand how to control. “The companies run on a startup mindset: Move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built,” said Jacob Coxon, a former engineer at Anthropic who has become a leading voice for regulation.
Coxon was a little-known engineer at Anthropic until he announced his departure through a series of social media posts that caught global attention. Coxon spent three years doing research at both Anthropic and OpenAI, said on the social platform X that the two AI companies are more focused on beating each other and global competitors in developing the most advanced model possible than they are on safety. He told the Council that AI could have tremendous upsides, but without transparency and safeguards, it’s more likely that things will spiral out of control. “On the current path, I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction,” Coxon said.
Days after Coxon resigned and aired his fears of worst-case scenarios, top executives including Anthropic’s Dario Amodei, OpenAI’s Sam Altman, SpaceXAI’s Elon Musk said they agreed AI’s development needs to slow down. AI companies have detailed steps they have taken to prioritize safety, but a coordinated industry slowdown faces several obstacles. Anthropic and OpenAI have said their advanced AI products need to be regulated, and company representatives at the hearing expressed openness to some of the Council’s suggestions. “I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared,” said Morgan Dwyer of OpenAI.
The City Council is considering several AI regulations, including proposals to require companies to disclose information about AI tools, require third-party approval for AI systems in the city and allow whistleblowers to receive a portion of fines from AI companies that violate laws.
Separately, New York Gov. Kathy Hochul, a Democrat, has moved to impose reporting and transparency standards around AI, as have governors elsewhere, though their power is somewhat limited since AI technology can reach across state lines, as well as the globe. Logan Graham, of Anthropic, told the Council the company welcomes “smart regulation” and that state and local governments have a role to play in developing frameworks. “AI is powerful enough that getting this wrong has real consequences, for individuals, institutions, public trust,” he said. “We have to work relentlessly to do the science of mitigating the risks.”
Source: AP
